1. Field of the Invention
The present invention generally relates to a field of a digital image processing, and in particular, to a technique for embedding or inserting identification data (electronic watermark data) having special information into a digital image.
2. Description of the Related Arts
Over the years, it has become a problem to copy digital images illegally. In order to prevent this illegal copy, it has been considered that digital image data are enciphered and only reproduction systems with a lawful cipher deciphering key can reproduce the enciphered digital image data. However, once the cipher is deciphered, this system can not prevent subsequent copies.
Therefore, in order to prevent an illegal use and copy of digital images, a method has been considered where special information (this information is referred to as electronic watermark data as described below) is embedded into the digital images themselves. As for such electronic watermark data for digital images, two kinds of data, such as visible and invisible electronic watermark data, are conceivable.
The visible electronic watermark data is made visually perceptible by synthesizing a special word or code for an image. Although such insertion of the visible electronic watermark data may cause deterioration of an image quality, it may visually evoke prevention of an illegal diversion to users of the digital images.
One example of embedment of visible electronic watermark data is disclosed in Japanese Patent Laid-Open Publication No. Hei 8-241403. In this approach, when visible electronic watermark data is synthesized into an original image, the electronic watermark data is synthesized into the original image by changing only a brightness component of pixels corresponding to opaque portions of the electronic watermark data by a scaling value without changing a color component. The scaling value is determined by a color component, a random number, and a value of pixels of the electronic watermark data.
In contrast, in the case of the invisible electronic watermark data, as the electronic watermark data is embedded into an image with care not to cause the image quality to be deteriorated, there is hardly any deterioration of the image quality originated from insertion of the electronic watermark data, so that the electronic watermark data embedded is visually imperceptible. If information capable of identifying an author is embedded into the image as this electronic watermark data, the author can be discriminated by detecting this electronic watermark data, even after an illegal copy has been performed. In addition, if information expressing disapproval of copying is embedded as this electronic watermark data, when a reproduction device detects the copy disapproval information, for example, the reproduction device can inform a user that the image is copy disapproval data or cause a copy prevention mechanism in the reproduction device to activate and restrict copying to a VTR or the like.
As one method for embedding invisible electronic watermark data into a digital image, there is a method in which special information as electronic watermark data is embedded into some portions with little effect to an image quality such as the least significant bit (LSB) or the like of pixel data. However, in this method, the electronic watermark data can easily be removed from the image. For example, if an image with the embedded electronic watermark data is subjected to a low-pass filtering, information of LSB of a pixel would be lost, so that the electronic watermark data can be removed without any appreciable deterioration. In addition, as an image compression processing reduces an amount of data by reducing an amount of information in portions with less effect on the image quality, the electronic watermark data can be removed without significant degradation of the image quality by causing such image data with the embedded electronic watermark data to be subjected to a compression and decompression process.
Alternatively, another example is disclosed in Japanese Patent Laid-Open Publication No. Hei 6-315131. This technique described in the publication detects an area where an image is not deteriorated even if the area is displaced by a peripheral area when the image is reproduced by utilizing a correlation of images in successive frames, and embeds specific information into the area by converting the level in the area to be converted. This approach detects the electronic watermark data and reconstructs the image by identifying the area into which identification data has been embedded by using a signal missing portion and conversion information and by correcting this identified area by using peripheral data in the identified area when the image is reproduced.
As yet another example, Japanese Patent Laid-Open Publication No. Hei 5-30466 discloses a method where a frequency of an image signal is converted and the information having lower frequency signal than a frequency range of the image signal whose frequency has been converted is embedded. This method takes out the original image signal using a high-pass filter and the identification data embedded using a low-pass filter.
As yet another example for frequency-converting an image, a method is proposed where an image is frequency-converted and electronic watermark data is embedded into an area with strong frequency components in an image signal after frequency conversion has been performed (Nikkei Electronics, Apr. 22, 1996, No. 660, page 13). This method embeds the electronic watermark data into the area with strong frequency components, so that, even if the image is subjected to an image process such as compression and decompression processes or filtering process, the electronic watermark data may not be lost. The electronic watermark data can not be removed, as long as the original data is destroyed. Furthermore, by adopting random numbers according to a normal distribution, an interference between the electronic watermark data may be prevented, and also by embedding, the deterioration of the image quality based on the embedment of the electronic watermark data may be precluded.
The method for embedding electronic watermark data in this method is that an original image is transformed into frequency components using DCT (discrete cosine transformation) or the like, n data of f(1), f(2), . . . , f(n) are selected which represent high values in a frequency domain, electronic watermark data of w(1), w(2), . . . , w(n) are selected from a normal distribution whose average is 0 and variance is 1, and the following equation is calculated for each i. EQU F(i)=f(i)+.alpha..times..vertline.f(i).vertline..times.w(i)
wherein .alpha. is a scaling factor.
The image with the embedded electronic watermark data can then be obtained by subjecting F(i) to the inverse DCT transformation.
Detection of the electronic watermark data is performed using the following method. In this detection method, the original images f(i) and electronic watermark data candidates w(i) (where i=1, 2, . . . , n) must be known.
Firstly, the image with the embedded electronic watermark data is caused to be transformed into frequency components using DCT, and element values corresponding to f(1), f(2), . . . , f(n) into which the electronic watermark data are embedded in the frequency domain are made F(1), F(2), . . . , F(n). The electronic watermark data W(i) are calculated and extracted by f(i) and F(i) using the following equation. EQU W(i)=(F(i)-f(i))/f(i)
Next, the statistical resemblance of w(i) and W(i) are calculated using a vector inner product and the following equation. EQU C=W.cndot.w/(WD/wD)
where:
W=(W(1), W(2), . . . , W(n)); PA1 w=(w(1), w(2), . . . , w(n)); PA1 WD=an absolute value of vector W; PA1 wD=an absolute value of vector w; and PA1 .cndot. is an inner product of a vector.
When the statistical resemblance C is larger than some specific value, it is estimated that the applicable electronic watermark data may be embedded.
If the electronic watermark data is embedded into an image using this method, it is effective when an author having the original image performs detection processing to a digital image which is assumed to be an illegal copy. This method needs the original image, so that the electronic watermark data can be detected when the author performs the detection processing to an image data which is assumed to be an illegal copy, however, in reproduction devices in each terminal can not detect the electronic watermark data because the reproduction devices have no original image.
Therefore, a method has been proposed where this method is improved for a terminal processing, particularly for an MPEG system. In the method, the original image is divided into blocks of 8 pixels.times.8 pixels, and the electronic watermark data is embedded into and extracted from the blocks as a processing unit.
The embedding process of the electronic watermark data will be performed using the following procedure. First, in a frequency domain after the discrete cosine transformation has been performed in the MPEG encoding process, f(1), f(2), . . . , f(n) are defined sequentially from lower frequency components in AC components, and electronic watermark data w(1), w(2), . . . , w(n) are selected from a normal distribution whose average is 0 and variance is 1, and the following equation is calculated for each i: EQU F(i)=f(i)+.alpha..times.avg(f(i)).times.w(i)
where .alpha. is a scaling factor and avg(f(i)) is a partial average which is the average of absolute values of three points in the neighborhood of f(i).
Then, the subsequent process of the MPEG encoding process is performed using F(i) instead of f(i).
The electronic watermark data will be detected using the following procedure. In this detection method, the original image is not needed, but only the electronic watermark data candidates w(i) (where i=1, 2, . . . , n) are needed to be known.
In a frequency domain of the blocks after an inverse quantization in an MPEG decompression or decoding process is completed, F(1), F(2), . . . , F(n) are defined sequentially from lower frequency components. An average of absolute values of three points of F(i-1), F(i), and F(i+1) in the neighborhood of F(i) is defined as a partial average avg(F(i)), and the electronic watermark data W(i) are calculated by the following equation: EQU W(i)=F(i)/avg(F(i))
and further sum WF(i) of an image are calculated for each i.
Next, the statistical resemblance of w(i) and WF(i) are calculated using inner products of vectors and the following equation. EQU C=WF.cndot.w/(WFD.times.wD)
When the statistical resemblance C is larger than some specific value, it is estimated that the applicable electronic watermark data may be embedded.
In the example disclosed in Japanese Patent Laid-Open Publication No. Hei 6-315131, the electronic watermark information are not embedded into all frames, so that illegal copy can not be prevented for frames without an electronic watermark embedded. Additionally, this example presupposes that successive frames are still images and there is no change in the successive frames, so that in cine images including a fast moving object, an area into which the electronic watermark data is embedded can not be identified, making it impossible to embed the electronic watermark data.
In addition, in the example disclosed in Japanese Patent Laid-Open Publication No. Hei 5-30466, the electronic watermark data is embedded into lower frequency domain than that of after an image has been frequency-transformed, so that the electronic watermark data can be easily removed using a high-pass filter.
Furthermore, in the example where the electronic watermark data is embedded into a strong portion in frequency components after an image has been frequency-transformed, the electronic watermark data can not be removed by a filter or the like. However, when plural electronic watermark data are embedded into one image, there is a problem that, if plural electronic watermark data are embedded into the same frequencies, the electronic watermark data are negated each other, thereby lowering a detectability factor at the time of detection.